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CROSS REFERENCE TO RELATED APPLICATIONS 

This application claims the benefit of United States Provisional Patent Application Serial No. 
60/217,997 filed July 13, 2000 entitled "MIXED-MODE INTERACTION/MOBILE PHONE BAR 
CODE ET AL," of common assignee herewith. 

FIELD OF THE INVENTION 

The present invention relates generally to wireless devices, and more particularly to wireless 
devices using mixed voice and data . 

BACKGROUND OF THE INVENTION 



In addition to their more traditional functions, mobile phones and other wireless devices are 
currently capable of obtaining and displaying information from the Internet or other informational 
networks. However, unlike computers which usually have fairly large display screens, mobile phones 
and other relatively small wireless devices do not provide access to the Internet that is as convenient as 
20 that provided by traditional computers. Small wonder, since traditional "web browsing" was developed 

with conventional sized computer displays in mind. 

In order to overcome the limitations imposed by the smaller display screens of earlier wireless 
devices, a number of options have been pursued by wireless device manufacturers and wireless service 

25 providers to provide more suitable Internet access. One of the more prevalent methods has been to 

make the display screen larger. Of course, given that the overall dimensions of wireless devices have 
decreased, physical constraints continue to limit the effectiveness of such efforts. Another method 
involves making more efficient use of available screen area by limiting the display to primarily textual 
objects. The combination of these two methods has met with some degree of success, but other 

30 obstacles to ease of use have not been overcome. 

For example, traditional web browsing sometimes requires a user to enter a uniform resource 
locator (URL) to access a particular web page. The small size of most wireless devices makes entering 
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text difficult, even if a keyboard-style keypad is provided. When only a standard telephone-style 
keypad is provided, entering text becomes that much more difficult. 

The inherent difficulties associated with using small interfaces and keypads to navigate web 
sites is a direct barrier to the widespread adoption of many Internet-type services. For wireless devices 
to become popular, an easier means for interacting with the "wireless web" would be advantageous. 
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SUMMARY OF THE INVENTION 



Therefore, what is needed is a way to leverage the voice functionality available on many 
wireless devices, and the data capability associated with those same devices. Accordingly, at least one 
5 embodiment of the present invention provides a method that includes receiving spoken input from a 

wireless communication device, and retrieving information associated with the spoken input. The 
method also includes delivering, to the wireless communication device, a non-verbal response to the 
spoken input. The non-verbal response is based on the retrieved information. In another embodiment, 
the input is non-verbal and the response is verbal 

10 

Another embodiment of the present invention provides a system comprising a server 
configurable to receive spoken input from a wireless communication device. The server is further 
configurable to retrieve information associated with the spoken input and to deliver, to the wireless 
communication device, a non-verbal response to the spoken input. The non-verbal response is based on 
1 5 the retrieved information. In another embodiment, the input is non-verbal and the response is verbal. 

Yet another embodiment of the present invention provides a method that includes obtaining, 
from a wireless communication device user, purchase notification criteria, obtaining information 
associated with purchasing opportunities, and selecting purchasing opportunities based, at least in part, 
20 on the purchase notification criteria. The method further includes notifying the user of selected 

purchasing opportunities via a wireless communication device, receiving spoken input from the 
wireless communication device in response to the notification, and delivering, to the user's wireless 
communication device, a non-verbal response to the spoken input. In another embodiment, the input is 
non-verbal and the response is verbal. 

25 

An advantage of at least one embodiment of the present invention is that obtaining information 
from the Internet is more convenient than using conventional wireless Internet access methods. 

Another advantage of an embodiment of the present invention is that making purchases from 
30 the Internet is easier than using conventional wireless purchasing methods. 

An additional advantage of at least one embodiment of the present invention is that merchants, 
suppliers and vendors can more effectively market distressed, time sensitive, location sensitive or 
general inventories of goods or services. 
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Yet another advantage of an embodiment of the present invention is that suppliers can more 
easily communicate with potential customers in a manner preferred by the customer, thereby increasing 
customer responsiveness. 

A further advantage is that various methods according to the present invention can be 
implemented using the voice and data capabilities of existing wireless devices. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Other objects, advantages, features and characteristics of the present invention, as well as 
methods, operation and functions of related elements of structure, and the combination of parts and 
5 economies of manufacture, will become apparent upon consideration of the following description and 

claims with reference to the accompanying drawings, all of which form a part of this specification, 
wherein like reference numerals designate corresponding parts in the various figures, and wherein: 

FIG. 1 is a block diagram illustrating various types of communication paths between a mobile 
1 0 subscriber and a server, according to one embodiment of the present invention; 

FIG. 2 is a block diagram of a system for conducting mixed-mode electronic commerce 
transactions according to one embodiment of the present invention; 

15 FIG. 3 is a flow diagram illustrating an "impulse" purchase transaction according to one 

embodiment of the present invention; and 

FIG. 4 is a block diagram representing a business model according to at least one embodiment 
of the present invention. 

20 
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DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 
OF THE INVENTION 

FIGS. 1-4 illustrate a mixed voice/data wireless system that can be used in conducting Internet 
5 commerce. As discussed in greater detail below, various embodiments of such a mixed-mode wireless 

system allow wireless customers to make voice requests from their wireless devices and receive data 
responses using the same, or different wireless devices. Alternatively, non-verbal requests can be made 
and verbal responses received. 

10 Referring first to FIG. 1, various mixed communication paths between a mobile subscriber 

110 and a web server 150 will be discussed. Mobile subscriber 110 preferably includes individuals 
who are using wireless devices having both voice and data capabilities, however in at least one 
embodiment a combination of one device having voice capabilities and another device having data 
capabilities can be used. Primary examples of devices having both voice and data capabilities are 

15 certain cellular and PCS telephones. Devices having only voice capabilities include older mobile 

phones, two-way radios and the like, while devices having data capability include pagers, personal 
digital assistants (PDAs), most common portable computers, and the like. 

In at least one embodiment, mobile subscriber 1 1 0 initiates communication with web server 
20 150 by issuing a voice request for information via voice request/response channel 112. Voice 

request/response channel 112 includes any necessary cellular or PCS stations, telephone lines, 
repeaters, routers, communications switches, etc. which are commonly used in making voice calls from 
a mobile phone or other wireless device. The voice request is received by the speech services 155 of 
web server 150, which translates the voice request into a data format more suitable for processing by 
25 web server 150. Web server 150, using data transformation services 240 or otherwise, retrieves the 

information requested by mobile subscriber 1 10 by accessing data sources 190. Data sources 190 are 
preferably accessible through a network, such as the Internet, but may include privately accessible data 
sources under the control of the same person or organization controlling web server 150. Once web 
server 150 has retrieved the necessary information from data sources 190, the information is delivered 
30 to mobile subscriber 1 1 0 in a non-verbal format. 

In other embodiments, mobile subscriber 1 1 0 initiates communication using a non-voice 
request submitted via user-initiated lookup channel 118, and voice request/response channel 112 is 
used to deliver a response to the non-voice request. The non-verbal request is processed by web server 
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150, and the results are converted to voice format using speech services 155. The voice formatted 
results are then provided to mobile subscriber 110 via voice request/response channel 112. Except 
where specifically noted, throughout the remainder of this document, reference to procedures, events, 
methods, etc., in which a mobile subscriber initiates communication via a voice request and receives a 
5 non-verbal response are intended to cover the converse situations in which the user initiates 

communication via non-verbal means and receives a voice response. 

As discussed above, a preferred embodiment of the present invention uses both verbal and 
non-verbal request/reply formats. A number of non-verbal request/reply formats can be used in 

10 implementing the present invention, and various embodiments employ different combinations of one or 

more suitable reply formats. One such reply format is a wireless markup language (WML) response. 
WML is a data format similar to hypertext markup language (HTML), except that WML is tailored for 
use in delivering Internet or other content to mobile phones and other similar wireless devices, while 
HTML is more commonly used to deliver content to conventional processing systems such as laptop 

15 computers, desktop computers and workstations. WML allows wireless devices conforming to the 

Wireless Application Protocol (WAP) to implement a web browser for display of information on a 
screen, in a manner similar to the way HTML compliant web browsers are used to display information 
on computers with larger screens. WML, like HTML, is tag-based, and supports text and image 
presentation, data input and forms. WAP compliant wireless devices commonly allow users to enter 

20 text by pressing number keys. In addition to WML and HTML, various embodiments of the present 

invention use modified HTML, Palm Query Access (PQA) or other suitable protocols. Note that in the 
following discussion, WML is used for exemplary purposes only, that the discussion applies equally to 
other suitable protocols including, but not limited to, those mentioned above. 

25 When it is desired to send a WML request, or a WML response to a voice request, the WML 

request/response is sent via a WML response channel 1 14, which includes a WAP gateway 160, which 
formats the response according to WAP protocols for use with a wireless network. In a preferred 
embodiment of the present invention, WAP gateway 160 is an existing third party gateway, various 
implementations of which can be utilized in implementing the present invention. If a version of WML, 

30 such as WML 1.1 requires an active browser session to exist, a user having only a single wireless 

device would need to terminate the voice call that was used to send the voice request over voice 
request/response channel 1 12, and initiate a browser session in order to receive the WML response. To 
accommodate this eventuality, delivery of the response could be delayed for a sufficient amount of time 
to allow the user to initiate the required browser session. Other versions of WML, such as WML1.2, 
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can provide the ability to deliver WML data without the need to initiate an active browser session. The 
use of a protocol that does not require an active browser session for message transmission, negates the 
need for a user to terminate a voice call before receiving a WML response. 

5 Another non-verbal request/reply format that can be employed according to one embodiment 

of the present invention is a short messaging service (SMS) message. SMS is a "push" based protocol 
that can push short text messages to mobile phones and similar devices. Push technology is a 
client/server based model in which a server initiates information exchange with a client. Push 
transactions are the opposite of pull transactions, in which clients initiate information exchanges with 
10 servers. Those skilled in the art will recognize that conventional web browsers employ primarily pull 

transactions to retrieve information from desired Internet addresses through the use of uniform resource 
locators (URLs). It should be noted, however, that even though SMS messages are examples of the 
push model, SMS messages may be used in conjunction with pull protocols. 

15 Consider, for example a mobile subscriber 110 who is driving through an unfamiliar city. 

Mobile subscriber 110 uses his mobile phone to make a voice request to receive information about 
restaurants that are near his present location. The voice request is delivered to web server 150, via 
voice request/response channel 1 12, and web server 150 uses speech services 155 to interpret the voice 
request. Web server 150 uses the interpreted voice request to gather information from data sources 

20 190, which may include Internet sites and/or other information sources. In one embodiment, the 

information gathered by web server 150 could be the URL of a local entertainment web site that 
includes the name and location of restaurants in the area. In such an embodiment, web server 150 
pushes the URL back to mobile subscriber 1 10 in an SMS message via an SMS message channel 116 
that includes SMS gateway 170. As with WAP gateway 160, SMS gateway 170 is preferably a third 

25 party system used in implementing various embodiments of the present invention. The user can then 

view the SMS message, and use the URL to launch a browser that would retrieve (pull) the web page 
indicated by the URL. In another embodiment, specific pieces of information gathered by web server 
150 are compiled into a text SMS message that is delivered to mobile subscriber 1 10 in place of or in 
addition to a URL. By providing specific pieces of information in an SMS text message, there is no 

30 need for a user to start a browser session to retrieve the desired information. 

Simple mail transport protocol (SMTP) is used to deliver a non-verbal response to mobile 
subscriber 110 in another embodiment. SMTP is one type of electronic mail (e-mail) protocol, and 
allows information gathered by web server 150 to be delivered to mobile subscriber 110 as an e-mail 
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message or attachment. The information is preferably formatted as a text e-mail message, and 
delivered through SMTP gateway 180 and SMS gateway 170 to mobile subscriber 110. In the above 
example, instead of initiating communications using a voice request, mobile subscriber 1 10 could have 
delivered an SMTP formatted e-mail request, and the web server 150 could have delivered a voice 
5 response to the SMTP request. 

In another embodiment of the present invention, mobile subscriber 110 sets a bookmark to 
easily access the URL of a results page. Mobile subscriber 110 delivers a voice request to web server 
150 via user-initiated lookup channel 118. Web server 150 gathers the information requested, and 
10 writes the information to a predetermined results page. Mobile subscriber 110 then retrieves the 

information from the predetermined results page using the bookmarked URL. The bookmarked URL 
can be activated by typing the URL into an address bar, selecting an icon displayed on a screen, 
activating a predetermined button or series of buttons on the mobile device being used by mobile 
subscriber 1 1 0, or by any other suitable method. 

15 

The preceding discussion has discussed FIG. 1 largely as a system in which mobile subscriber 
1 10 provides a voice request, and web server 150 responds with a non-verbal reply. As noted earlier, 
however, the present invention may also be used "in reverse," such that mobile subscriber 110 provides 
a non-verbal request for information using a suitable protocol, such as SMTP, SMS or WML as 
20 previously described, and web server 150 provides a verbal response to mobile subscriber 1 10 via voice 

request/response channel 112. Numerous suitable combinations of voice and data interactions and 
protocols between mobile subscriber 110 and web server 150 may be implemented according to the 
principles set forth herein. 

25 Referring next to FIG. 2, a functional diagram of a mixed-mode system is illustrated, and 

designated generally by reference numeral 200. Services 250 that mixed-mode system 200 can use in 
implementing various embodiments of the present invention include, but are not limited to, voice wallet 
252, rules engine 254, database 256, and location services 259. HTTP server 220 accesses services 250 
through data transformation service 240 and message bus 270. Data transformation service 240 is also 
' 30 used to interface with notification server 230, and external commerce servers 260. User devices 210, 

which have been discussed in relationship to mobile subscriber 1 10 in FIG. 1, can interface with HTTP 
server 220 directly, or through speech services 155. Additionally, notifications can be delivered to user 
devices 210 through speech services 155 or directly from notification server 230 using various non- 
voice protocols, as previously discussed. Mixed-mode system 200 can receive content through content 
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input 215. It should be noted that notifications, both verbal and non-verbal, are preferably session 
oriented. By providing session oriented notifications, mobile subscriber 100 (FIG. 1) can "drill-down" 
through multiple menu levels. 

5 In one example of how mixed-mode system 200 can be used, mobile subscriber 1 10 (FIG. 1) 

sends a voice request from user devices 210 to HTTP server 220, via speech services 155. User 
devices 210 preferably include mobile phones or other user devices capable of both voice and data 
communications. However, in at least one embodiment, one device is a voice capable device, for 
example an older cellular telephone, and another device is data only enabled, such as some PDAs. 
10 These two devices can be used in combination to implement a mixed-mode system capable of sending 

voice requests and receiving data replies, or sending data requests and receiving voice replies. 

Speech services 155 may include any suitable combination of hardware and software needed 
for speech recognition and/or translation. In one embodiment, speech services 155 is used to translate 
15 spoken requests into suitably formatted data for use by HTTP server 220. In other embodiments, 

speech services 155 is used to translate replies from HTTP server 220 into speech for delivery to a 
voice capable user device 210. Speech services 155 may also be used in conjunction with voice wallet 
252, as subsequently discussed. 

20 Content received through content input 2 1 5 can be either voice content that is routed to HTTP 

server after translation by speech services 155, or data content delivered directly to HTTP server 220. 
The content can include recorded audio and/or visual content for display, various types of information 
to be stored in database DBMS 256, or any other suitable type of content that may provide value or aid 
in implementing the present invention. 

25 

HTTP server 220 employs services 250 to fulfill requests from user devices 210. HTTP server 
220 accesses services 250 through data transformation service 240, which in turn uses message bus 
270. Data transformation service 240 supplies application program interfaces (APIs) for creating 
extensible markup language (XML) documents, as well as converting between different XML 
30 document dialects that may be employed by external commerce systems 260, or other external systems 

and devices. Various suitable scripting languages may be employed to perform these functions 
consistent with the objects of the present invention. 
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External commerce systems 260 include Internet web sites and web pages, private systems 
maintained by companies for inventory, scheduling, transportation, or other similar systems. External 
commerce systems 260 may be utilized to allow mobile subscriber 110 (FIG. 1) to make purchases 
directly from merchants in control of external commerce systems 260. Alternatively, external 
5 commerce systems 260 may supply information for storage in database DBMS 256 or for use by other 

services 250. Message bus 270 is used to add publish/subscribe capabilities to mixed-mode system 
200, in addition to providing greater interoperability with external systems, improved internal 
modularity within mixed-mode system 200, and enhanced scalability. 

10 Services 250 are used to provide value to mobile subscribers 110 (FIG. 1) employing user 

devices 210 to make queries and receive responses. In at least one embodiment, voice wallet 252 
provides a secure, yet easily accessible voice authenticated storage area for sensitive purchasing 
information. Voice wallet 252 allows for storage of credit card numbers, expiration dates, personal 
identification numbers (PINs), and other information that may be used for purchasing transactions. 

15 Once a user stores purchasing information in his voice wallet 252, he does not need to enter the 

information again. To conduct purchase transactions, the user need only speak a predetermined phrase 
to recall any necessary transaction information, thereby eliminating the need to enter information each 
time a new purchase transaction is desired. 

20 In at least one embodiment, the predetermined phrase is a user's phone number, or other non- 

secure phrase. To initialize security on his voice wallet 252, the user simply speaks the phrase, and the 
phrase is recorded. When the user desires to access his voice wallet 252, he need only speak the 
predetermined phrase into user device 210, which transmits the spoken phrase through speech services 
155 to voice wallet 252. Voice wallet 252 compares the spoken phrases characteristics with the 

25 characteristics of the recorded phrase to determine if the speaker is authorized access to voice wallet 

252. Using a voice authentication system to secure sensitive credit card information can relieve the 
user of tedious data entry, while providing an extremely high level of security. 

Another service, rules engine 254 works in conjunction with information stored in database 
30 DBMS 256, and information received from external commerce system 260 and content input 215. 

Rules engine 254 correlates consumer interests with inventory attributes such as price, location and 
quantity, and with dynamic attributes such as time and threshold. The rules engine may be considered 
to be dual sided, inasmuch as one side performs evaluations of the interest rules and the other side 
evaluates and executes notification rules. In a preferred embodiment, each rule contains two sets of 
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information. The first set of information is a 1 to n list of conditions and their Boolean relationships. 
The second set of information is a 1 to n list of actions to be taken when particular rules are evaluated 
as true. A very basic example is a rule that specifies sending an SMS message listing local produce 
suppliers with excess inventory in response to a standing request for notification of produce available 
for purchase at three o'clock every business day. 

Much of the information upon which rules engine 254 operates is stored in database DBMS 
256. DBMS 256 preferably includes user profiles database 257 and inventory database 258. 
Additional and/or different databases may be employed consistent with the objects of the present 
invention. Inventory database 258 may include information received from external commerce servers 
260 and content input 215. User profiles database 257 preferably includes information associated with 
user preferences and purchasing patterns, shipping addresses, payment information, and the like. Some 
of this information is preferably supplied by mobile subscriber 110 (FIG. 1) through user devices 210, 
and other information is gleaned from past user actions. It will be appreciated that user profiles 
database 257 can include any suitable information that may be useful in providing mobile subscriber 
1 10 pertinent information regarding available purchasing options or otherwise. 

Inventory database 258 preferably includes information regarding available merchandise, 
services, accommodations, or other items available for purchase. Preferably, inventory database 258 is 
updated frequently by external commerce systems 260, so that the information in inventory database 
258 accurately reflects currently available purchasing opportunities. In at least one embodiment of the 
present invention, information in user profiles database 257 is compared against currently available 
purchasing opportunities stored in inventory database 258 using rules engine 254 to determine when, 
and what type of notification should be sent to particular mobile subscribers 1 10. 

In addition to the information stored in database DBMS 256, rules engine 254 may use 
information from location services 259 to determine when a notification should be sent to a particular 
one of user devices 210. Location services 259 are responsible for location determination and 
searching functions. These functions include, but are not limited to, geocoding addresses, route 
plotting, dynamic positioning, geodetic database searches, and the like. Once a mobile subscriber's 
position has been determined, rules engine 254 can use the position information to identify purchasing 
opportunities that are geographically relevant to the mobile subscriber 1 10. Location services 259 can 
be used in conjunction with information from inventory database 258, external commerce systems 260 
or content input 215 to ensure that only relevant information is delivered to mobile subscriber 1 10. For 
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example, if a mobile subscribed 10 is on a business trip, and submits a request to find a tailor, the user 
probably has no use for the phone number of a tailor in his home city, rather, he is likely to be 
interested only in finding a tailor near his current location. It will be appreciated that various locating 
mechanisms, including the use of global positioning satellite coordinates, user supplied coordinates, 
5 and location information available from cellular and PCS telephone systems, can be used without 

departing from the spirit and scope of the present invention. 

Referring next to FIG. 3, a method of using a mixed-mode system to conduct commerce will 
be discussed according to one embodiment of the present invention. The method begins at step 310, in 

10 which purchase notification criteria is obtained. Preferably, purchase notification criteria include 

specific circumstances or events identified by the user, the occurrence of which should trigger a 
notification. For example, a collector may specify that he wants to be notified of any Elvis 
memorabilia being offered for sale in a set geographical area around the collectors home. In other 
embodiments, additional notification criteria obtained or derived from sources other than the user may 

15 be employed consistent with the objects of the present invention. In addition to selecting conditions 

for notification, the collector may specify the type of notification he wishes to receive. For example, if 
the Elvis memorabilia is priced over $1,000 dollars, he may desire to receive an SMTP e-mail message 
on his mobile phone. If, however, the memorabilia is priced below $100 dollars, the collector may 
wish to receive an SMS message indicating contact information for the seller. In at least one 

20 embodiment, the notification may take the form of a voice notification via a mobile phone. Other 

notification methods have been discussed with reference to FIG. 1 . 

Once the purchase notification criteria have been obtained in step 310, purchasing opportunity 
information is obtained in step 320. Purchasing opportunity information includes, but is not limited to, 
25 description of the item for sale, price, availability, location, and similar information. Purchasing 

opportunity information may be stored in a database, such as inventory database 258 in FIG. 2, or may 
be obtained via the Internet or otherwise in a near-real-time fashion. 

The method continues with step 330, in which purchasing opportunities are selected from the 
30 purchasing opportunities obtained in step 320. This selection is preferably based on the purchasing 

notification criteria obtained in step 310. In effect, step 330 looks for purchasing opportunities 
supplied by merchants that match notification criteria supplied by consumers. When a match is found, 
the method proceeds to step 340, and the mobile subscriber is notified of selected purchasing 
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opportunities that meet the notification criteria. Various methods that can be used to provide user 
notification of purchasing opportunities have already been discussed earlier. 

In at least one example, when the notification is sent to a user, the user's mobile device may 
5 provide an audible, visual or tactile alert to make the user aware that a notification has been received. 

The alert may be a special ringing or vibrating pattern, or any other suitable alert, including a standard 
ring, beep, etc. The user may respond to the notification to which he has just been alerted with a voice 
request to take advantage of one or more of the selected purchasing opportunities. In at least one 
embodiment, either voice or data notifications establish notification session, which incorporate drill- 
1 0 down type menus, so that the user can obtain more detailed or specific information. 

In step 350 the user's voice request is received and processed by mixed-mode system 200 
(FIG. 2). The processing may include accepting a purchase order, providing additional information 
regarding purchasing opportunities, altering the notification criteria, or otherwise. In many cases, a 
15 non-verbal response is delivered to the user's wireless device in response to the spoken request, in the 

same manner as the original notification was delivered. 

Note that while a particular embodiment of a method according to the present invention has 
been discussed, numerous suitable variations may be employed in implementing the present invention. 

20 For example, in step 350, the user may provide non-verbal input, such as depressing a "yes/no" button 

or a designated sequence of keys, instead of providing spoken input. In addition, step 360 may deliver 
a verbal response instead of a non-verbal response. Additionally, the original notification provided in 
step 340 could be delivered in the form of a voice call that notifies the user of purchasing opportunities. 
Finally, various steps may be omitted or added. For example, in one embodiment, the user sends a 

25 voice and/or data request for information without first being prompted by a notification. The request 

may or may not be for information related to a purchase, and various embodiments of the present 
invention may be used for accessing the Internet or performing other purchase or non-purchase related 
tasks according to the mixed-mode principles set forth herein. 

30 Referring next to FIG. 4, a basic business model according to one embodiment of the present 

invention will be discussed. Consumer 410 is assumed to be concerned with three main purchasing 
factors: scheduling 412, location 414, and interests 416. Scheduling 412 takes into account whether or 
not a purchasing opportunity is available when needed. For example, if a consumer 410 needs to 
purchase a new suit before an important meeting in two weeks, then a sale beginning a month later does 
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not meet the scheduling concerns of consumer 410. Location 414 is related to the current location of 
consumer 4 1 0; if consumer 4 1 0 is in California, he is unlikely to be interested in a renting a car in New 
York. Finally, interests 416 are related to user preferences. If consumer 410 indicates that he is 
interested in purchasing a multi-family rental property on the North side of town, he is unlikely to 
5 appreciate being notified every time a single family dwelling on the South side of town comes on the 

market. However, when a particular purchasing opportunity satisfies all three of these factors, 
consumer 410 is likely to consider taking advantage of that purchasing opportunity. 

One purchasing factor that immediately comes to mind in most cases is a price criterion. It 
10 will be appreciated that while not specifically illustrated, price may be included as an element of 

interests 416. Continuing with the previous example, if the consumer wishes to purchase a multi- 
family rental property in a particular price range, then notifying the consumer of opportunities that are 
significantly out of that range would likely be undesirable. Various methods of specifying a price 
range of interest can be employed without departing from the spirit and scope of the present invention. 
1 5 One example of how price ranges can be used is, by asking a user to select a "price tolerance 

preference" during an initial configuration phase, and presenting to the user opportunities having prices 
within this specified "price tolerance." It will be appreciated that interests 416 are not limited to price 
interests; instead interests 416 include any suitable user preferences that may be associated with 
particular items, services, objects and the like. 

20 

Partner 420 has goods, services, real-estate, information, or other items available for purchase. 
Partner 420 is concerned with selling factors including product schedule 413, product location 415 and 
products 416. Product schedule 413 relates to the availability of the product for sale, and product 
location 4 1 5 relates to where the product is physically located. For example, if partner 420 is a 
25 nationwide chain of automobile dealerships with a large inventory of unsold four-wheel drive trucks in 

Atlanta, but has a limited inventory of the same four-wheel drive trucks in Dallas, partner 420 will be 
most interested in attempting to sell the excess vehicles in Atlanta. The nature and quantity of products 
41 6 to be sold are also a factor to be considered. 

30 In at least one embodiment, a service provider employs a system 450 to correlate the purchase 

factors that are important to consumer 410, with the selling factors used by partner 420. When selling 
factors align with purchase factors, a notification of purchasing opportunities can be provided to the 
consumer. As illustrated in FIG. 4, system 450 is expandable to provide integration with both presently 
available resources, as illustrated by consumer input/output technologies 460 and partner input output 
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technologies 465, and expected future technological advances, such as future packet-based migration 
440 and future locator technology 430. 

In summary, consider the following examples of particular mixed-mode transactions 
5 according to different embodiments of the present invention. In a first embodiment, an individual is at 

a shopping mall, and notices a surfboard on sale for "50% off." Wanting to know if the advertised sale 
price is really a bargain, he uses his mobile phone to place a voice request for surfboard prices. The 
request is received and translated by a system similar to the one described in FIG. 2. The system 
access its own databases, and finds some relevant information. Next the system searches the Internet, 

10 and retrieves information about surfboard prices from a dedicated surfboard resale web site. The 

system combines the information from the website and the database, and formats the information into 
an SMS text message. The SMS text message is used to send the combined information to the 
individual's mobile phone, which beeps upon receipt of the message to alert the user. The mobile 
phone user then views the information received, but wants more information about a particular surf 

15 board listed in the message. In at least one embodiment, the SMS text message includes drill-down 

menu capability. One way of providing drill-down capability is by including a WML tag in the SMS 
text message, thereby allowing the user to access a URL where more information can be found. The 
user selects the WML tag, which causes his mobile phone to initiate an active browser session and 
automatically direct him to the web site identified by the URL. The user now has all the information he 

20 needs to make an informed decision. 



In a second example, consider a mobile subscriber 110 as discussed in FIG. 1. In this 
example, mobile subscriber 1 1 0 is an executive of a food preparation corporation seeking new freezing 
equipment to use in the freezing and preparation of frozen dinners. Since mobile subscriber 1 10 is 

25 interested in equipment used in frozen food preparation, he configures notification criteria specifying 

that when any cryogenic freezers become available for sale, he is to be notified by receipt of both an e- 
mail notification and a voice notification. Meanwhile, a small startup company begins manufacturing a 
new type of cryogenic freezer, and is looking for buyers. The small company partners with a provider 
of mixed-mode wireless systems, and allows the provider to advertise the new freezer to interested 

30 parties. The provider, through the use of various rule systems discussed earlier, matches the 

executive's notification criteria with the partner's selling criteria, and determines that a notification 
should be made. 
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The executive is called on his mobile phone, and a computer generated voice message is 
delivered, notifying him of the opportunity to purchase the freezer. At the same time, an e-mail 
message containing relevant information is delivered to his mobile phone. The executive may respond 
to either the voice notification or the e-mail message using either verbal or non-verbal means. As 
5 noted earlier, both the verbal notification and the non-verbal notification are preferably session 

oriented, thereby providing drill-down menu capabilities. Assuming that the executive responds to the 
voice notification by indicating that he wishes to purchase the freezer, he may be asked to speak a 
predetermined phrase to access his voice wallet, thereby allowing the purchase to be completed. 
Confirmation of the purchase can be sent to the executive's mobile phone using various messaging 
1 0 and/or voice formats. 



In this third example, consider a mother running errands with her children. On her way home 
after a hectic shopping day, she realizes that she has forgotten to get one item; she needs to find a 
particular brand of pants in a hard to find size. Rather than going from store to store trying to find the 

1 5 needed pair of pants, she places a call on her web capable mobile phone and places a verbal request for 

the location of a store that has the needed pants in stock. She hangs up the phone, and moments later 
her phone beeps, alerting her that she has received and SMS message. The SMS message, generated as 
discussed earlier, has a location map of a store in her vicinity that keeps the desired pants in stock. She 
may then drive to the store and pick up the item. Alternatively, she may send a response to the SMS 

20 message (either verbal or via the web capabilities of her mobile phone) and request that the store put a 

hold on the pants. The store may, in turn, reply to the request for a hold by calling the mother on her 
mobile phone (or sending another SMS message), confirming that the pants are indeed on hold. 

Consider now a final example of how an embodiment of the present invention may be 
25 employed in assisting a sales representative to more effectively manage distressed inventory. Assume 

that a sales representative for a candy company has a large lot of candy that is approaching the end of 
its storage life, and needs to be sold quickly. Just prior to attending an out-of-town conference 
meeting, the sales representative uses his wireless enabled mobile phone to verbally request a report on 
the number of units of candy sold in the past twenty-four hours. During the meeting, the sales 
30 representative receives a non-verbal response to his request for a sales report. The sales representative 

notes that sales are not proceeding as quickly as he would like, and so he sends a non-verbal reply to 
the non-verbal response, indicating that the candy's price should be reduced by ten percent. 

In the preceding detailed description, reference has been made to the accompanying drawings 
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which form a part hereof, and in which are shown by way of illustration specific embodiments in which 
the invention may be practiced. These embodiments, and certain variants thereof, have been described 
in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood 
that other suitable embodiments may be utilized and that logical, mechanical, chemical and electrical 
5 changes may be made without departing from the spirit or scope of the invention. To avoid 

unnecessary detail, the description omits certain information known to those skilled in the art. The 
preceding detailed description is, therefore, not intended to be limited to the specific forms set forth 
herein, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents, as 
can be reasonably included within the spirit and scope of the appended claims. 
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WHAT IS CLAIMED IS: 



1 L A method comprising: 

2 receiving spoken input from a wireless communication device; 

3 retrieving information associated with the spoken input; and 

4 delivering, to the wireless communication device, a non-verbal response to the spoken input, 

5 the non-verbal response based on the retrieved information. 

1 2. The method as in Claim 1, wherein the step of delivering includes using a Wireless Markup 

2 Language to deliver the non-verbal response. 

1 3 . The method as in Claim 1 , wherein the step of delivering includes using a Short Messaging 

2 Service message to deliver the non-verbal response. 

1 4. The method as in Claim 1, wherein the step of delivering includes using Simple Mail 

2 Transport Protocol to deliver the non-verbal response. 

1 5. The method as in Claim 1, wherein the non-verbal response includes the retrieved information. 

1 6. The method as in Claim 1, wherein the non-verbal response includes a Uniform Resource 

2 Locator. 

1 7. The method as in Claim 1, wherein the non-verbal response includes a Handheld Device 

2 Markup Language link. 
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1 8. A system comprising a server configurable to: 

2 receive spoken input from a wireless communication device; 

3 retrieve information associated with said spoken input; and 

4 deliver, to said wireless communication device, a non-verbal response to said spoken input, 

5 said non-verbal response based on said retrieved information. 

1 9. The system as in Claim 8, further including a Wireless Applications Protocol gateway, said 

2 server configurable to deliver said non-verbal response via said Wireless Applications Protocol 

3 gateway. 

1 10. The system as in Claim 8, further including a Short Messaging Service gateway, said server 

2 configurable to deliver said non-verbal response via said Short Messaging Service gateway. 

1 11. The system as in Claim 8, further including a Simple Mail Transport Protocol gateway, said 

2 server configurable to deliver said non-verbal response via said Simple Mail Transport Protocol 

3 gateway. 

1 12, The system as in Claim 8, wherein said non-verbal response includes said retrieved 

2 information. 

1 13. The system as in Claim 8, wherein said non-verbal response includes a Uniform Resource 

2 Locator. 

1 14. The system as in Claim 8, wherein said non-verbal response includes a Handheld Device 

2 Markup Language link. 
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1 15. A method comprising: 

2 obtaining, from a wireless communication device user, purchase notification criteria; 

3 obtaining information associated with purchasing opportunities; 

4 selecting purchasing opportunities based, at least in part, on the purchase notification criteria; 

5 notifying the user of selected purchasing opportunities via a wireless communication device; 

6 receiving spoken input from the wireless communication device in response to the 

7 notification; and 

8 delivering, to the user's wireless communication device, a non-verbal response to the spoken 

9 input. 

1 1 6. The method as in Claim 15, wherein the step of delivering includes using a Wireless Markup 

2 Language to deliver the non-verbal response. 

1 1 7. The method as in Claim 15, wherein the step of delivering includes using a Short Messaging 

2 Service message to deliver the non-verbal response. 

1 1 8. The method as in Claim 15, wherein the step of delivering includes using Simple Mail 

2 Transport Protocol to deliver the non-verbal response. 

1 19. The method as in Claim 15, wherein the non-verbal response includes information associated 

2 with the selected purchasing opportunities. 

1 20. The method as in Claim 15, wherein the non-verbal response includes a Uniform Resource 

2 Locator. 

1 21 . The method as in Claim 15, wherein the non-verbal response includes a Handheld Device 

2 Markup Language link. 
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1 22. A method comprising: 

2 receiving non-verbal input from a wireless communication device; 

3 retrieving information associated with the non-verbal input; and 

4 delivering, to the wireless communication device, a verbal response to the non-verbal input, 

5 the verbal response based on the retrieved information. 

1 23 . The method as in Claim 22, wherein the step of delivering includes using a Wireless Markup 

2 Language to deliver the non-verbal response. 

1 24. The method as in Claim 22, wherein the step of delivering includes using a Short Messaging 

2 Service message to deliver the non-verbal response. 

1 25. The method as in Claim 22, wherein the step of delivering includes using Simple Mail 

2 Transport Protocol to deliver the non-verbal response. 

1 26. The method as in Clairn22, wherein the non-verbal response includes the retrieved 

2 information. 

1 27. The method as in Claim 22, wherein the non-verbal response includes a Uniform Resource 

2 Locator. 

1 28. The method as in Claim 22, wherein the non-verbal response includes a Handheld Device 

2 Markup Language link. 



22 



Impulsity02 



PATENT APPLICATION 



A method comprising: 

obtaining, from a wireless communication device user, purchase notification criteria; 
obtaining information associated with purchasing opportunities; 

selecting purchasing opportunities based, at least in part, on the purchase notification criteria; 
notifying the user of selected purchasing opportunities via a wireless communication device; 
receiving non-verbal input from the wireless communication device in response to the 
notification; and 

delivering, to the user's wireless communication device, a verbal response to the spoken input 
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30. A system comprising a server configurable to: 

receive non-verbal input from a wireless communication device; 

retrieve information associated with said non-verbal input; and 
deliver, to said wireless communication device, a verbal response to said non-verbal input, said verbal 
response based on said retrieved information. 
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MIXED MODE INTERACTION 

ABSTRACT OF THE DISCLOSURE 

A user of a wireless device, such as a mobile phone, can make purchases or obtain information 
via a network, such as the Internet, using both voice and non-verbal methods. Users can submit voice 
queries and receive non-verbal replies, submit non-verbal queries and receive voice replies, or perform 
similar operations that marry the voice and data capabilities of modern mobile communication devices. 
The user may provide notification criteria indicating under what conditions a notification should be sent 
to the user's wireless device. When purchasing opportunities matching the selected notification criteria 
become available, the user is notified. The user can respond to the notification, and immediately take 
advantage of the purchasing opportunity if he so desires. Mixed-mode interactions can also be used by 
sellers to more advantageously control the marketing of distressed, time sensitive, or other 
merchandise/services. 
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RULES 63 AND 67 (37 C.F.R. 1.63 and 1.67) 
DECLARATION AND POWER OF ATTORNEY 

FOR UTILITY/DESIGN/CIP/PCT NATIONAL APPLICATIONS 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name; 



and 



I believe that I am the original, first and sole inventor (if only one name is listed below) 
or an original, first and joint inventor (if plural names are listed below) of the subject matter 
which is claimed and for which a patent is sought on the invention entitled: MIXED-MODE 
INTERACTION, the specification of which: (mark only one) 

X (a) is attached hereto. 
(b) was filed on as Application Serial No. and 

was amended on (if applicable) 

(c) was filed as PCT International Application No. PCT/ on and 

was amended on (if applicable). 

(d) was filed on as Application Serial No. and was issued a 

Notice of Allowance on . 

(e) was filed on and bearing attorney docket number . 



I hereby state that I have reviewed and understand the contents of the above identified 
specification, including the claims as amended by any amendment referred to above or as allowed 
as indicated above. 

I acknowledge the duty to disclose all information known to me to be material to the 
patentability of this application as defined in 37 CFR § 1.56. If this is a continuation-in-part 
(CIP) application, insofar as the subject matter of each of the claims of this application is not 
disclosed in the prior United States application in the manner provided by the first paragraph of 
35 U.S.C. § 1 12, 1 acknowledge the duty to disclose to the Office all information known to me 
to be material to patentability of the application as defined in 37 CFR § 1.56 which became 
available between the filing date of the prior application and the national or PCT international 
filing date of this CIP application. 

I hereby claim foreign priority benefits under 35 U.S.C. § 119/365 of any foreign 
application(s) for patent or inventor's certificate listed below and have also identified below any 
foreign application for patent or inventor's certificate filed by me or my assignee disclosing the 
subject matter claimed in this application and having a filing date (1) before that of the 
application on which my priority is claimed or, (2) if no priority is claimed, before the filing date 
of this application: 
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PRIOR FOREIGN PATENTS 

Date first laid- Date 

Month/Day/Year open or patented or Priority Claimed 

Number Country Filed Published Granted Yes No 



I hereby claim the benefit under 35 U.S.C. § 120/365 of any United States application(s) 
listed below and PCT international applications listed above or below: 

PRIOR U.S. OR PCT APPLICATIONS 

Application No. (series code/serial no.) Month/Day/ Year Filed Statusfpending, abandoned, patented) 



I hereby appoint Raymond M. Galasso, Reg. No. 37,832 and Edward J. Marshall, Reg. 
No. 45395 as my attorney and/or agents, with full power of substitution and revocation, to 
prosecute this application, provisionals thereof, continuations, continuations-in-part, divisional, 
appeals, reissues, substitutions, and extensions thereof and to transact all business in the United 
States Patent and Trademark Office connected therewith, to appoint any individuals under an 
associate power of attorney and to file and prosecute any international patent application filed 
thereon before any international authorities, and I hereby authorize them to act and rely on 
instructions from and communicate directly with the person/assignee/attorney/ firm/organization 
who/which first sent this case to them and by whom/which I hereby declare that I have consented 
after full disclosure to be represented unless/until I instruct them in writing to the contrary. 

Please address all correspondence and direct all telephone calls to: 

Raymond M. Galasso 

Simon., Fakhoury, Tangalos, Frantz & Galasso, PLC 

P.O. Box 26503 

Austin, Texas 78755-0503 

Telephone: (512) 372-8240 

Facsimile: (512) 372-8247 

I hereby declare that all statements made herein of my own knowledge are true and that 
all statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1 00 1 of Title 1 8 of the United States 
Code, and that such willful false statements may jeopardize the validity of the application or any 
patent issued thereon. 
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Date 
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